Analysis

Cleaning Data

.mat to .csv file for fasten analysis process

• select elemDataI and save into .mat format
• select necessary variables for analysis

Artist / Menu Task Detection

Filter Disposition file

• check if simulation contains three instances
• if both non-restart and restart file contains 3 instances, used non-restart file (Participant 7 YP, Participant 21 ZM, Participant 123 ZM)

Find Engagement point

We used AMOC method in changepoint library with assumption that there is single potential changepoint from time when task was available to participant to when task was correctly completed.

We determined changepoint with mean and variance of speed and lane deviation. Then, selected minimum timeframe (faster changepoint detection result) as predicted engagement point for artist task.

• no predicted engagement point was after the participant was incorrect

Example

exception Two simulations had one artist/menu task engaged (score didn’t change when task was available): Participant 123 YP (20140111102408.csv) and Participant 129 XM (20140104090136.csv)

Define Control vs Experiment Group

Start of experiment group: using task availabe time:

using predictedengagement time:

Visuals

We found that total time (from task available to complete) is longer as participant experience more task:

## # A tibble: 3 x 6
##   eventNum   min    Q1 median    Q3   max
##   <fct>    <int> <dbl>  <dbl> <dbl> <int>
## 1 1           86  180.   282.  602.   619
## 2 2          161  366.   512.  603    612
## 3 3          245  450.   603   604    617

Standard deviation of lane deviation by dosage group and secondary task engagement:

Conclusions:

• Not much difference when deriving engagement segment using changepoint detection vs. using entire period task is available
• We can still analyze the data both ways, but will use the changepoint detection approach for now

Boxplot version:

Conclusions:

• Standard Deviation of Lane Deviation is much higher when engaged in secondary task (artist-menu search)
• Similar differences across experimental conditions
• This visual ignores pairing of segments/participants

Paired (trt/control) version:

Conclusions:

• Within each experiment condition individual subject performance is worse when engaged in secondary task

Within Subject Performance Differences by Condition

Conclusions:

• Some (but not all) of the instances where performance was better while engaged in the secondary task had slower speeds during the task
• Difficult to gauge condition effects from this plot

Tables

Artist/Menu task segments available for analysis for each of the 19 subjects:

	XM	XP	YM	YP	ZM	ZP	Sum
Subject ID 3	3	3	3	3	3	3	18
Subject ID 7	3	3	3	3	3	3	18
Subject ID 10	3	3	3	3	3	3	18
Subject ID 15	3	3	3	3	3	3	18
Subject ID 17	3	3	3	3	3	3	18
Subject ID 18	3	3	3	3	3	3	18
Subject ID 21	3	3	3	3	3	3	18
Subject ID 25	3	3	3	3	3	3	18
Subject ID 26	3	3	3	3	3	3	18
Subject ID 29	3	3	3	3	3	3	18
Subject ID 31	3	3	3	3	3	3	18
Subject ID 32	3	3	3	3	3	3	18
Subject ID 34	3	3	3	3	3	3	18
Subject ID 35	3	3	3	3	3	3	18
Subject ID 104	3	3	3	3	3	3	18
Subject ID 113	3	3	3	3	3	3	18
Subject ID 120	3	3	3	3	3	3	18
Subject ID 123	3	3	3	2	3	3	17
Subject ID 129	2	3	3	3	3	3	17
Sum	56	57	57	56	57	57	340

Conclusions:

• 19 subjects, only 2 missing events

Marginal Statistics

Marginally, there is a statistically significant effect of secondary task engagement (for most Dose/Event combos) on standard deviation of lane deviation (p-values are from paired t-tests within subject/event):

DosingLevel	EventID	mean_SDL	sd_SDL	n	p_value	signif
XM	1	-0.1079823	0.6653311	19	0.47929
XM	2	0.3852318	1.0004674	18	0.10234
XM	3	0.6290988	0.9803059	19	0.00515	**
XP	1	0.3818299	0.8399112	19	0.04753	*
XP	2	0.1759859	1.4630276	19	0.60005
XP	3	0.4446652	0.8482441	19	0.02231	*
YM	1	0.2406506	0.9370421	19	0.26295
YM	2	-0.1298222	1.3342822	19	0.67149
YM	3	0.4220633	0.7426300	19	0.01324	*
YP	1	0.4575804	0.8007496	19	0.01274	*
YP	2	0.5653850	1.1621832	18	0.03902	*
YP	3	0.3885585	0.7604960	19	0.02594	*
ZM	1	0.2998543	1.0046808	19	0.19328
ZM	2	0.0252731	1.0177404	19	0.91380
ZM	3	0.6257512	0.8121137	19	0.00078	***
ZP	1	0.4451658	1.0555926	19	0.06603	.
ZP	2	0.5386620	1.0042601	19	0.01939	*
ZP	3	0.4225396	0.6069259	19	0.00241	**

Models

Model 1 - Base Model

\[y_{ij} = \beta_0 + \gamma_i + \beta_1 SecTask_{ij} + \beta_2 DoseXM_{ij} + \beta_3 DoseYM_{ij} + \ldots + \epsilon_{ij}\]

• \(y_{ij}\) = Standard Deviation of lane deviation for subject \(i\) at dosing level \(j\)
• \(\gamma_i\) = subject specific random intercept
• \(SecTask_{ij}\) = dummy variable indicating observation measures performance during secondary task (1 = sec task, 0 = control segment)
• \(DoseAA_{ij}\) = dummy variable for dose = “AA” (1 = specified dose, 0 = reference group)
  • In our models we will re-define the reference group such that all dose effects are positive

d3cp$Dose_Grp <- relevel(d3cp$DosingLevel, ref = "ZP")
fit <- lmer(data = d3cp, SD.Lane.Deviation ~ Dose_Grp +  Experiment  + (1 | ID))
fit_sum <- summary(fit)
kable(round(fit_sum$coefficients, 4))

	Estimate	Std. Error	df	t value	Pr(>|t|)
(Intercept)	0.4598	0.0853	171.3512	5.3935	0.0000
Dose_GrpXM	0.0154	0.1012	655.1849	0.1522	0.8790
Dose_GrpXP	0.0756	0.1007	654.9280	0.7502	0.4534
Dose_GrpYM	0.0873	0.1007	654.9280	0.8665	0.3865
Dose_GrpYP	0.0517	0.1012	655.1849	0.5109	0.6096
Dose_GrpZM	0.0454	0.1007	654.9280	0.4507	0.6524
Experiment	0.3443	0.0583	654.9280	5.9029	0.0000

kable(round(anova(fit), 4))

	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
Dose_Grp	0.6434	0.1287	5	655.0996	0.2226	0.9528
Experiment	20.1476	20.1476	1	654.9280	34.8437	0.0000

Conclusions:

• Significant effect of secondary task engagement on standard deviation of lane deviation

• No effect of experimental condition

• A few problems with this model:
  • Doesn’t adjust for differences in speed
  • Skewed distribution of SD.Lane.Deviation
  • Treats all 3 secondary tasks as the same
  • It also isn’t looking for effect modification by Dose_Grp

Model 2 - Log-transformed outcome, control for eventNum and Avg.Speed

fit <- lmer(data = d3cp, log2(SD.Lane.Deviation) ~ relevel(DosingLevel, ref = "XM") + 
              factor(eventNum)  + Avg.Speed + Experiment + (1 | ID))
kable(round(summary(fit)$coefficients, 4))

	Estimate	Std. Error	df	t value	Pr(>|t|)
(Intercept)	-3.9636	0.5686	370.8770	-6.9712	0.0000
relevel(DosingLevel, ref = “XM”)XP	0.1522	0.1954	652.9692	0.7789	0.4363
relevel(DosingLevel, ref = “XM”)YM	0.2630	0.1976	657.7072	1.3311	0.1836
relevel(DosingLevel, ref = “XM”)YP	0.1300	0.1964	653.7328	0.6617	0.5084
relevel(DosingLevel, ref = “XM”)ZM	0.3465	0.1972	657.0216	1.7573	0.0793
relevel(DosingLevel, ref = “XM”)ZP	0.0550	0.1969	656.3927	0.2792	0.7802
factor(eventNum)2	0.4422	0.1426	664.1713	3.1003	0.0020
factor(eventNum)3	0.3166	0.1417	662.7813	2.2341	0.0258
Avg.Speed	0.0288	0.0087	402.0684	3.3167	0.0010
Experiment	0.7464	0.1124	652.2426	6.6397	0.0000

kable(round(anova(fit), 4))

	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
relevel(DosingLevel, ref = “XM”)	9.1783	1.8357	5	655.6375	0.8545	0.5115
factor(eventNum)	21.6956	10.8478	2	659.0886	5.0497	0.0067
Avg.Speed	23.6309	23.6309	1	402.0684	11.0003	0.0010
Experiment	94.7050	94.7050	1	652.2426	44.0854	0.0000

Conclusions:

• Secondary task engagement, speed, and eventID are all important predictors of standard deviation of lane deviation
• Experimental condition is not significant when controlling for these other predictors

Model 3 - Log-transformed outcome, control for eventNum and Avg.Speed, test for effect modification

• Does Dose_Grp modify the performance deteroration seen during secondary tasks?

fit <- lmer(data = d3cp, log2(SD.Lane.Deviation) ~ relevel(DosingLevel, ref = "ZP") + 
              factor(eventNum)  + Avg.Speed + Experiment + 
              relevel(DosingLevel, ref = "ZP"):Experiment + (1 | ID))
kable(round(summary(fit)$coefficients, 4))

	Estimate	Std. Error	df	t value	Pr(>|t|)
(Intercept)	-3.9386	0.5590	387.3314	-7.0460	0.0000
relevel(DosingLevel, ref = “ZP”)XM	0.0083	0.2779	649.4002	0.0297	0.9763
relevel(DosingLevel, ref = “ZP”)XP	0.0782	0.2778	650.6645	0.2814	0.7785
relevel(DosingLevel, ref = “ZP”)YM	0.2091	0.2754	647.3339	0.7593	0.4479
relevel(DosingLevel, ref = “ZP”)YP	0.1210	0.2795	651.8014	0.4329	0.6652
relevel(DosingLevel, ref = “ZP”)ZM	0.4550	0.2754	647.2304	1.6523	0.0990
factor(eventNum)2	0.4430	0.1430	659.1465	3.0971	0.0020
factor(eventNum)3	0.3175	0.1421	657.7581	2.2334	0.0259
Avg.Speed	0.0286	0.0087	398.7192	3.2839	0.0011
Experiment	0.8304	0.2754	647.2321	3.0158	0.0027
relevel(DosingLevel, ref = “ZP”)XM:Experiment	-0.1252	0.3911	647.2303	-0.3201	0.7490
relevel(DosingLevel, ref = “ZP”)XP:Experiment	0.0399	0.3894	647.2418	0.1025	0.9184
relevel(DosingLevel, ref = “ZP”)YM:Experiment	-0.0024	0.3894	647.2402	-0.0063	0.9950
relevel(DosingLevel, ref = “ZP”)YP:Experiment	-0.0902	0.3911	647.2304	-0.2306	0.8177
relevel(DosingLevel, ref = “ZP”)ZM:Experiment	-0.3269	0.3894	647.2665	-0.8394	0.4015

kable(round(anova(fit),4))

	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
relevel(DosingLevel, ref = “ZP”)	8.1640	1.6328	5	649.3536	0.7556	0.5821
factor(eventNum)	21.7850	10.8925	2	654.0749	5.0409	0.0067
Avg.Speed	23.3021	23.3021	1	398.7192	10.7838	0.0011
Experiment	94.6682	94.6682	1	647.2411	43.8109	0.0000
relevel(DosingLevel, ref = “ZP”):Experiment	2.5570	0.5114	5	647.2534	0.2367	0.9463

Conclusions:

• Dose_Grp doesn’t appear to modify the effect of secondary task engagement

Model 4 - Paired Difference as Outcome

fit <- lmer(data = df, diff_SD_lane_pos ~ relevel(DosingLevel, ref = "YM") + factor(EventID) + diff_speed + (1 | SubjectID))
kable(round(summary(fit)$coefficients, 4))

	Estimate	Std. Error	df	t value	Pr(>|t|)
(Intercept)	0.1210	0.1562	211.2718	0.7743	0.4396
relevel(DosingLevel, ref = “YM”)XM	0.1231	0.1778	312.8555	0.6922	0.4893
relevel(DosingLevel, ref = “YM”)XP	0.1565	0.1769	312.6196	0.8849	0.3769
relevel(DosingLevel, ref = “YM”)YP	0.2848	0.1778	312.7221	1.6015	0.1103
relevel(DosingLevel, ref = “YM”)ZM	0.1390	0.1776	312.9546	0.7822	0.4347
relevel(DosingLevel, ref = “YM”)ZP	0.2910	0.1770	312.6593	1.6443	0.1011
factor(EventID)2	-0.0315	0.1305	315.2556	-0.2413	0.8095
factor(EventID)3	0.2020	0.1279	314.3777	1.5790	0.1153
diff_speed	-0.0004	0.0178	330.3307	-0.0235	0.9813

kable(round(anova(fit),4))

	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
relevel(DosingLevel, ref = “YM”)	3.3992	0.6798	5	312.8148	0.7622	0.5776
factor(EventID)	3.6341	1.8170	2	314.1180	2.0372	0.1321
diff_speed	0.0005	0.0005	1	330.3307	0.0006	0.9813

Conclusions:

• Little substantive difference analyzing the data this way

Model 5 - Avg Speed as Outcome

fit <- lmer(data = d3cp, Avg.Speed ~ relevel(DosingLevel, ref = "YP") + 
              factor(eventNum)  + Experiment   +  (1 | ID))
kable(round(summary(fit)$coefficients, 4))

	Estimate	Std. Error	df	t value	Pr(>|t|)
(Intercept)	63.3018	1.2485	34.3184	50.7011	0.0000
relevel(DosingLevel, ref = “YP”)XM	-1.5575	0.8088	653.0613	-1.9257	0.0546
relevel(DosingLevel, ref = “YP”)XP	-0.2623	0.8051	653.0257	-0.3258	0.7447
relevel(DosingLevel, ref = “YP”)YM	-5.2056	0.8051	653.0257	-6.4659	0.0000
relevel(DosingLevel, ref = “YP”)ZM	-4.9229	0.8051	653.0257	-6.1148	0.0000
relevel(DosingLevel, ref = “YP”)ZP	-4.6509	0.8051	653.0257	-5.7769	0.0000
factor(eventNum)2	4.2126	0.5693	653.0241	7.3998	0.0000
factor(eventNum)3	4.0525	0.5666	652.9927	7.1522	0.0000
Experiment	0.0939	0.4640	652.9927	0.2025	0.8396

kable(round(anova(fit),4))

	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
relevel(DosingLevel, ref = “YP”)	3347.6698	669.5340	5	653.0148	18.2942	0.0000
factor(eventNum)	2589.6467	1294.8233	2	653.0136	35.3794	0.0000
Experiment	1.5003	1.5003	1	652.9927	0.0410	0.8396

Conclusions:

• Dosing condition has a highly significant impact on participant speed
• Secondary task engagement doesn’t predict speed
• No interaction between dosing and secondary task engagment (model testing this not shown, p > .9)

Side Mirror Task

	XM	XP	YM	YP	ZM	ZP	Sum
Subject ID 3	11	14	13	14	14	14	80
Subject ID 7	14	14	14	15	14	14	85
Subject ID 10	13	14	14	10	14	13	78
Subject ID 15	14	12	14	14	11	14	79
Subject ID 17	14	14	14	14	12	14	82
Subject ID 18	15	14	14	17	14	14	88
Subject ID 21	13	11	13	12	17	11	77
Subject ID 25	14	14	14	14	14	12	82
Subject ID 26	14	13	14	14	13	12	80
Subject ID 29	14	14	13	14	14	8	77
Subject ID 31	13	13	10	13	14	14	77
Subject ID 32	10	11	9	14	14	11	69
Subject ID 34	14	14	13	14	13	14	82
Subject ID 35	15	12	12	14	13	14	80
Subject ID 104	12	13	12	14	14	14	79
Subject ID 113	14	14	14	14	12	14	82
Subject ID 120	14	13	14	14	13	14	82
Subject ID 123	14	2	14	14	14	10	68
Subject ID 129	13	13	14	14	14	13	81
Sum	255	239	249	263	258	244	1508